Effects of visual information on the face masked speech communication in Covid-secure classrooms

Acoustical measurements and speech intelligibility tests were carried out to investigate the effects of masks on speech communication experienced in real Covid-secure university classrooms during the pandemic. Face-masked speech levels and noise levels were measured to understand the acoustical effects of masks on speech sounds during 15 multiple lectures in 3 university classrooms. The speech intelligibility scores were also evaluated for lower and higher SNR (signal-to-noise ratio) conditions, and for with and without the presence of visual information conditions to investigate the effects of both the acoustic and visual signals in understanding speech communication in actual classroom situations. In the 3 active university classrooms the students experienced on average: speech levels of 55.1 dBA (σ = 5.5 dBA), noise levels of 42.3 dBA (σ = 3.9 dBA), and a speech-to-noise ratio of 12.8 dBA σ = 5.2 dBA). The mean SNR values at the listener's position for the 15 lectures varied from 3.6 dBA to 20.0 dBA. The use of a portable sound amplification system increases the face-masked speech levels mostly at mid and high frequencies (500–4 kHz), thus it can be more useful for achieving higher SNR values in classrooms. The presence of visual cues have little effect on achieving more higher speech intelligibility scores in higher SNR conditions. The present results show that visual obstruction of the talker's mouth decreases speech intelligibility scores by a maximum of 10% in lower SNR conditions, particularly at a SNR of 6 dBA or lower. © 2023 Elsevier Ltd

A Participatory Design Approach to Explore Design Directions for Enhancing Videoconferencing Experience for Non-signing Deaf and Hard of Hearing Users

The breakout of the COVID-19 pandemic shifted people's daily activities from in-person to video-mediated ones. Many people with hearing loss encounter cognitive overload due to ineffective visuals of the videoconferencing interface and therefore find meeting contents difficult to comprehend. This research incorporates a participatory design methodology to investigate the Deaf and Hard of Hearing (DHH) users' tacit needs. DHH users demonstrated ways of mitigating their hardships in the workshop, such as emphasizing the visual hierarchy or assigning visual cues to fixed positions. These findings are used in developing design directions for creating a more inclusive online environment. © 2022 Owner/Author.

Facemask occlusion's impact on L2 listening comprehension & nbsp;

Wearing a facemask impacts oral communication as it is both a barrier to the acoustic signal and occludes nonverbal cues such as lip movements and facial expressions. However, while past studies have suggested that these factors do not cause significant impediments to comprehension among speakers of the same first language, the current study investigates the impact facemasks have in the context of second language learners. N = 192 participants were divided into three groups of n = 64 and asked to listen to an 89 s speech. To isolate the effects of visual cues on listening comprehension, the same audio recording was used for all experimental groups. Condition One was a video of a speaker with no mask. Condition Two was a video of the same speaker wearing a mask. Condition Three was an audio recording. The significant finding was that participants in the second (masked) condition scored significantly lower on subsequent comprehension quizzes than the other two. Implications to language instruction during the COVID-19 pandemic will be discussed.& nbsp;

Multi-Modal Sarcasm Detection in Social Networks: A Comparative Review

Sentiment Analysis (SA) has become an extremely sought after area of research especially post COVID-19 when people used to spent a lot of time on the social media to interact with each other. This interaction was done through posts having both textual and visual cues and also by participating in online discussions forums. Some of the inherent challenges encountered in the process of SA include discernment of sarcasm, irony, humor, negation, multi-polarity or Aspect-Level Sentiment Analysis (ASA) etc. Researchers are now gradually shifting their focus to the identification and detection of sarcasm and how it can empower SA. Sarcasm expresses a person's downside feelings by using positive words in an implicit way. It also has an overall impact on increasing the efficiency of the SA models. Eliciting sarcastic statements is tough for humans as well as for machines without the knowledge of the context or background in which it is expressed, body language and/or facial expression of the speaker and his voice modulation. This review paper studies some of the approaches used for sarcasm detection and also guides researchers in exploring the different modalities of data for developing applications like a virtual chat-bot or assistant, depression analysis, stress management system at workplace etc. © 2022 IEEE.

A Taxonomy for Augmented and Mixed Reality Applications to Support Physical Exercises in Medical Rehabilitation-A Literature Review.

In the past 20 years, a vast amount of research has shown that Augmented and Mixed Reality applications can support physical exercises in medical rehabilitation. In this paper, we contribute a taxonomy, providing an overview of the current state of research in this area. It is based on a comprehensive literature review conducted on the five databases Web of Science, ScienceDirect, PubMed, IEEE Xplore, and ACM up to July 2021. Out of 776 identified references, a final selection was made of 91 papers discussing the usage of visual stimuli delivered by AR/MR or similar technology to enhance the performance of physical exercises in medical rehabilitation. The taxonomy bridges the gap between a medical perspective (Patient Type, Medical Purpose) and the Interaction Design, focusing on Output Technologies and Visual Guidance. Most approaches aim to improve autonomy in the absence of a therapist and increase motivation to improve adherence. Technology is still focused on screen-based approaches, while the deeper analysis of Visual Guidance revealed 13 distinct, reoccurring abstract types of elements. Based on the analysis, implications and research opportunities are presented to guide future work.

Perceived Listening Difficulties of Adult Cochlear-Implant Users Under Measures Introduced to Combat the Spread of COVID-19.

Following the outbreak of the COVID-19 pandemic, public-health measures introduced to stem the spread of the disease caused profound changes to patterns of daily-life communication. This paper presents the results of an online survey conducted to document adult cochlear-implant (CI) users' perceived listening difficulties under four communication scenarios commonly experienced during the pandemic, specifically when talking: with someone wearing a facemask, under social/physical distancing guidelines, via telephone, and via video call. Results from ninety-four respondents indicated that people considered their in-person listening experiences in some common everyday scenarios to have been significantly worsened by the introduction of mask-wearing and physical distancing. Participants reported experiencing an array of listening difficulties, including reduced speech intelligibility and increased listening effort, which resulted in many people actively avoiding certain communication scenarios at least some of the time. Participants also found listening effortful during remote communication, which became rapidly more prevalent following the outbreak of the pandemic. Potential solutions identified by participants to ease the burden of everyday listening with a CI may have applicability beyond the context of the COVID-19 pandemic. Specifically, the results emphasized the importance of visual cues, including lipreading and live speech-to-text transcriptions, to improve in-person and remote communication for people with a CI.

Impacts of face coverings on communication: an indirect impact of COVID-19.

OBJECTIVE: To understand the impact of face coverings on hearing and communication. DESIGN: An online survey consisting of closed-set and open-ended questions distributed within the UK to gain insights into experiences of interactions involving face coverings, and of the impact of face coverings on communication. SAMPLE: Four hundred and sixty members of the general public were recruited via snowball sampling. People with hearing loss were intentionally oversampled to more thoroughly assess the effect of face coverings in this group. RESULTS: With few exceptions, participants reported that face coverings negatively impacted hearing, understanding, engagement, and feelings of connection with the speaker. Impacts were greatest when communicating in medical situations. People with hearing loss were significantly more impacted than those without hearing loss. Face coverings impacted communication content, interpersonal connectedness, and willingness to engage in conversation; they increased anxiety and stress, and made communication fatiguing, frustrating and embarrassing - both as a speaker wearing a face covering, and when listening to someone else who is wearing one. CONCLUSIONS: Face coverings have far-reaching impacts on communication for everyone, but especially for people with hearing loss. These findings illustrate the need for communication-friendly face-coverings, and emphasise the need to be communication-aware when wearing a face covering.